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The  J-divorgenco  is  considered  as  a measure 
of  discrimination  information  in  continuous-time 
observation  for  a binary  hyivitliosis  testing  prob- 
lem. Relations  l:>etweon  the  sampling  rate,  the 
observation  time  interval  and  the  amount  of  in- 
formation are  discussed  from  a detection  view 
point.  General  asymptotic  relations  are  obtain- 
ed as  well  as  some  finite  observation-time  re- 
sults for  stationary  Gaussian  processes. 

K 

I.  INTRODUCTION  ^ 


Denote  by  J(1,0)  the  J-divergence  between  the  dis- 
tributions of  the  sampled-obsorvation  vector  under 
the  hypothesis  and  It  is  defined  by 


Jtl,0)  = /{f^(x)  - (1) 

where  denotes  the  probability  density  function 
of  the  observation  X under  hyi>othesis  It  is 

a measure  of  tlie  degree  of  "rloscnoss*'  of  two  dis- 
tributions, and  is  closely  related  to  the  dc^tec- 
tion  probability  (4). 


Studies  on  ronlinuous-t ime  waveform  samp- 
ling problems  have  been  focused  mainly  oi.  the 
reconstruction  of  the  original  waveform  accord- 
ing to  the  minimum  mear.-squaro-error  criterion. 

For  detection  puriXises,  hov/ever-  where  wc  are 
interested  in  dctcrminifig  whether  a signal  is 
present  in  the  observation  or  not,  the  goodness 
of  the  estimate  of  the  waveform  itself  from  sam- 
ples may  not  be  the  most  relevant  aspect  of  the 
problem.  In  II],  for  example,  it  has  been  shown 
that  for  detection  of  weak  signals  in  additive 
noi.se  the  cliaractoristic  of  the  optimum  quanti- 
zer approximates  the  nonlinearity  p*/p,  where  p 
and  p are  the  noise  density  function  and  its 
derivative;  p* (x)/p(x)  may  not  be  equal  to  x. 
Therefore  it  is  of  interest  to  have  results  on 
the  degree  of  degradation  of  detector  perform- 
ance due  to  sampling,  results  which  would  en- 
able one  to  find  a reasonable  sampling  rate  af- 
fording an  acceptable  c:o.T»pronise  between  the 
reduction  of  volume  of  computation  and  data,  and 
oi>timization  of  detection  performance.  In  this 
paper  we  consider  the  relations  between  the  sam- 
pling rate,  the  observation-time  interval,  and 
discrimination  information  in  samples  from  a 
continuous-time  observation. 

Consider  a simple  binary  detection  problem 
of  testing  a null  hypothesis  and  an  alterna- 
tive Hj,  using  uniformly  sampled  observations 

from  a statiot>ary  moan-square  continuous  Gaus- 
sian process.  A stationary  Gaussian  process  is 
considered  since  it  plays  an  important  role  in 
many  tyjxis  of  problems,  and  allows  analytical 
results  to  be?  obtained.  As  a measure  of  discrim- 
ination information  in  an  observation  vector  of 
samples  the  J-divergonce  (?,3)  v/ill  be  employed. 
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In  the  following  section  the  problems  of  de- 
tection of  a con.siant  signal  and  a time-varying 
signal  in  additive  noise  are  considerc?d.  For  Ixith 
ca.ses  the  J-divergonce  in  sonie  finite  numljor  N of 

samples,  denoted  by  is  obtained  and  compared 

T 

with  the  total  J-divergence,  denoted  by  J , for 
tlie  entire  continuous-time  observation  in  tho 
finite  observation  time  interval  T.  This  enables 
us,  for  instance,  to  pick  a reasonable  sampling 
rate  which  will  yield  samples  containing  a cer- 
tain percentage  of  the  total  information  avail- 
able. In  the  asymptotic  case  v/here  both  T and  N 
tend  to  infinity,  with  T/N  remaining  fixed  at 
some  finite  value,  exact  analytic  expressions  can 
N T 

be  obtained  for  J /j  . Detection  probabilities 
for  some  given  probability  of  false  alarm  have 
also  been  computed  to  give  an  example  of  the  re- 
lation between  the  J-divergence  and  the  detection 
probability. 

It  is  interesting  to  note  tliat  tlio  results 
obtained  indicate  that  when  the  problem  is  the 
detection  of  a constant  signal,  for  instance,  and 
the  noise  is  banJlimitod  with  bandwidth  W,  a num- 
ber of  samples  which  slightly  oxcced.s  WT  is  all 
that  we  need  for  detection  purposes,  rather  than 
the  2WT  samples  taken  at  the  Ny^iist  rate. 

Specifically,  the  two  hypotheses  and 

that  we  will  consider  for  our  continuous-time  ob- 
servation X(t)  are  defined  as  follows: 


Hjt  X(t)  - s(t)  + N(t) 
X(t)  - N(t) 


, -T/2<t<T/2 
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More,  s(t)  denotes  a known  finite-energy  signal 
and  N(t)  denotes  a stationary  Gaussian  process 
with  an  autocorrelation  function  K(i),  r being 
the  time  difference  between  two  observations. 
Components  X.  of  the  observation  vector  X are 
defined  as 


^i+1  “ “ 2^' 


where  A denotes  the  time  difference  between  any 
two  consecutive  samples,  so  that 

A = T/(N-1)  . (4) 

We  will  use  the  term  "J-Ratio"  to  denote  the 
N T 

ratio  of  d“divergeucc,  0 /J  . By  total  J-diver- 
gence,  J , we  mean 


,T  , . ,N 
J « lim  J 


for  fixed  T, 

II.  J-DTVnRGENCE  AND  SAMPLING  RATE 
T 

We  will  first  show  that  J can  be  given  by 
the  J-divergonce  of  the  coefficients  of  the 
Karhu  nen-Loeve  expansion  of  the  process  X(t). 
Then  the  expression  for  asymptotic  J-Ratio  will 
be  derived. 

A.  N~Sample  J-divergence 

We  obtain  first  the  J-divcrgence  in  N uni- 
formly sampled  observations  in  the  observation 
time  interval  of  length  T.  We  will  assume  that 
the  noise  is  a zero-mean  process  to  simplify  our 
discussion.  Let  be  the  covariance  matrix  of 

the  N-samples.  Then  we  can  write 


K(0) 

K(A) 

K(T) 

K(&) 

K(0) 

K(T-A) 

K(T) 

K(T-A)  •• 

K(0) 

which  is  a symmetric  Tooplitz  matrix  (5).  I,et 
<t).^  denote  the  matrix  whose  columns  are  the  nor- 
malized eigenvectors  of  R^  and  let  denote  the 

eigenvalues  of  R.  . k*=l,2, . . . ,N.  We  note  that 
N 

hh  ' Vi 

where  ^ is  the  i-th  column  of  By  transform- 

ing X to  X such  that 

x'  , (8) 

and  because  is  an  invertible  mapping  [6], 


we  can  easily  see  that  J is  given  by 

i=l  \ 


where  is  the  signal  component  of  the  j-th  sam- 
ple and  is  the  ji-th  clement  of  4^  , i.e.  tlie 
31  N 

j-th  component  of  the  eigenvector 
B.  Total  J-div€;rqonre 

IjOt  us  denote  the  Karhuncn-Ix)eve  (KL)  expan- 
sion of  x(t)  in  the  interval  (-T/?,T/2]  by 


X(t)  = I Y.  (t) 
i=i 


where  the  coefficients  Y.  are 
1 

f T/2 


f T/Z 

Y « X(t).i/.  (t)dt. 

^ J-.T/2  ^ 


The  orthonormal  basis  functions  ^.*s  satisfy  the- 
following  integral  equation: 


= I" 


K(t-i)l(/j^(i)dT  (12) 


We  know  that  are  orthogonal  and  thus  inde- 

pendent random  variables,  since  they  are  normal- 
ly distributed.  Defining  S.  to  be 


(■T/2 

5.  ■=  s(t)  Ip.  ( 

> -T/2 


WO  got  the  J-divergence,  J , in  the  coefficients 
i=l,2, . . . , to  be 

_ 2 


KL  ” ^i 
J = E — 

i=l  "i 


The  integral  equation,  (12),  holds  for  all  t, 
-T/2ct<T/2.  We  can  approximate  the  integral  in 
(12)  by  a finite  sum  and  consider  the  equations 
for  t = -T/2,A-T/2, , . . ,T/2.  Then  the  set  of 
equations  can  be  written  as  an  approximate  ma- 
trix equation  of  finite  dimensions, 

a , (15) 

where  is  a vector  whose  (j+l)-th  comjjonent  is 
t(ij(jfi-T/2)  for  j=0,l, . . . ,N-1.  This  becomes  the 

s.ime  equation  as  we  had  in  (7)  if  wc  let  N-*«>  for 
fixed  T.  Therefore,  with  <1’^^  the  j-th  component 


I jOTWlpl/tniUBJlTCito  . 

IDir’  ii4/W 


oo 


2 


A more  rigorous  derivation  of  (21)  is  possible 
along  the  lines  in  tlie  Appendix. 


of  ij>. , 


■ F''  ^ ^ 1 

I41I  f s . Ij;.  . I 

N * Li=l  5 J 

lim  J = lira  1 — j- 

N~>  A'O  i=l  '"i 


A-'O  i=l 
T fixed 

fT/2 


= 7. 

i=l 


[C- 


(16) 


which  is  the  J-divergenco  in  the  KL  expansion  co- 
efficients. 

N T 

C.  Asymptotic  Ratio  of  J and  J 


In  many  cases  where  T is  finite,  finding  a 
N 

closed-form  expression  for  J may  not  be  simple. 
Numerical  computations  also  have  a limit  with  in- 
creasing number  of  sari'i>les  since  they  involve, 
in  general,  a matrix  inversion.  However,  in  the 
asymptotic  case  where  both  T and  N tend  to  infin- 
ity with  A remaining  fixed  at  some  finite  value, 

N T 

wo  can  examine  the  ratio  of  J and  0 as  a func- 
tion of  A. 


For  T>>1,  the  eigenvalues  and  the  eigen- 
functions satisfying  the  integral  equation,  (12), 
may  be  approximated  by  [7], 

^ N^(nfo)  (17) 


and 


li'(t) 


, n=0,±l,±2... 


(l/i^)exp(-j2'rrfQnt) 


(18) 


where  f^^l/T  and  N^(f)  is  the  power  spectrum  of 
the  noise  process  given  by 


N (f) 
c 


K(i)e“^^''^’'dT. 


(19) 


(Note  that,  in  (17)  and  (18),  the  eigendata  are 
indexed  over  both  ixjsitivc  and  negative  integers 
(7).)  Also,  define 


S^(f) 


s(t)c  dt, 


(20) 


the 

For 

and 


Fourier  transform  of  the  energy  signal  s(t) 
T>>1,  using  the  apj)roximation  given  by  (17) 
(18)  in  (16),  wc  get  _ 

ls^(f)|2 


lim  J 


L 


N (f) 
c 


df 


(21) 


Let  us  now  define 


N^(f)  = r.  K(nA)e"^^''^'’'^  i (22) 


then  J 

K(mA)  = A N^(f)e^^"^”^df . (23) 

' 2A 

Similarly,  define  S^{f)  for  signal  .s(t).  After 
some  manipulation  we  can  show  tliat 


lim  J 
N,T->«> 

A fixed 


N 


A 


Nd(f) 


df 


(24) 


The  derivation  of  (24)  is  outlined  in  the  Appen- 
dix. From  (21)  and  (24)  we  got  tne  asymptotic 

N T 

ratio  of  J and  J , 


Asymp.  J-Ratio 


(25) 


In  the  constant  signal  case,  assuming  finite  en- 
ergy by  letting  s(L)  * s->0  as  the  asyniptot- 
ic  J-Katio  reduces  to 


N (0) 

Asymp.  J-Ratio  = ^ N"(b) 
d 

By  the  Poisson  sum  formula  [8)  we  cai  write 

N.(f)  = J 7 N (f  + n/A) . (27) 

d A c 

n=— * 

Therefore,  if  the  noise  spcctrvun  has  a bandwidth 
W,  the  ratio  given  by  (26)  remains  unity  as  long 
as  1/A>W.  In  otlier  words  a samidincj  rate  higher 
than  half  of  the  Ny<|pist  rate  should  yield  sam- 
ples which  contain  full  discrimination  informa- 
tion in  terms  of  J-divergence . In  time-varying 
signal  cases  if  the  signal  is  frequency  limited 

with  a bandwidth  W and  with  W <W,  then  a sam- 
s s ' 

pling  rate  of  (W*W  ) should  be  sufficient  for  the 
detection  problem. 

From  this  asymptotic  characteristic  of  J- 
divergences  with  respect  to  sampling  rate  we  can 


3 


1 


1 

! 


see  that  the  doqradation  of  detector  perfomuinco 
due  to  samplinq  at  a rate  below  the  Nyguist  rate 
could  be  insiynif icant.  If  wo  plot  the  J-Ratio 
for  a specific  detection  problem  of  interest,  we 
can  use  it  as  a guide  to  picking  a sampling  rate. 
Wo  would  then  have  a reasonable  theoretical  jus- 
tification for  this  choice  in  terms  of  the  per- 
centage of  full  discrimination  information  it  pre- 
serves in  the  samples. 

III.  NUMERICAL  EXAMPLES 

In  the  following  examples  we  assume  until 
eticrqy  signals  for  convenience. 

Example  1.  Constant  Signal  and  Ideal  low  Pass 
No  3 se 

To  see  if  we  have  characteristics  for  the 
J-Ratio  in  a finite  number  of  samples  similar  to 
what  we  noticed  in  the  asymptotic  case,  we  con- 
sider an  example  for  constant  signal  detection 
where  the  additive  noise  has  the  pcjwcr  spectral 
density  as  shown  in  rig.  1.  In  order  to  obtain 
the  finite-T  J-Katio  and  the  probability  of  de- 
tection, the  observation  time  duration  T is  as- 
sumed to  be  lb.  Tims  the  number  of  samples 
taken  at  the  Ny^ist  rate  is  31.  The  detectioi. 
probability,  P^,  is  computed  for  = 10“'®  whore 

Pp^  denotes  the  false  alarm  probability.  In  Fig. 
1,  the  asymptotic  and  the  finite-T  J-Ratio  and  P^^ 
arc  plotted.  Wc  note  that  both  J-Ratio  and 

show  a jump  to  their  maximum  values  as  N exceeds 
16  and  remain  at  the  value  for  N>17. 

Example  2.  Constant  Signal  and  Gauss-Markov 
Noi  so 


When  the  noise  is  Gauss-Markov  where  the  co- 
variance  function  is  of  the  form 

-air’ 

K(T)  = K^e  ' (28) 

analytic  derivation  of  J^,  J*^  and  the  asymptotic 
J-Ratio  is  possible.  We  list  the  results  below. 
For  convenience,  we  let  Kq*=1. 


= (N(l-p)  + 

2pl/(Hp) 

(29) 

= aT/2  + 1 

(30) 

Asymp.  J-Ratio 

= (l-e"®*)/[aA(l+e'®'' 

)) 

(31) 

where 

„-aT/(N-l) 
p = e 

(32) 

For  a=2  and  T=15, 

those  are  plotted  in 

Fig. 

2. 

It  shows  the  close  relationship  between  the  J- 
divergence  and  the  detection  probability.  Sup- 
pose WG  are  to  find  a sampling  rate  such  that  the 
samples  obtained  at  the  rate  preserve  90%  of  the 
total  discrimination  information,  we  can  read  it 
from  the  asymptotic  J-Ratio  curve  which  is,  in 
this  case,  AsO.56. 


Example  3.  Time-varying  Signal  and  Ideal  l^w  i 

Pass  Noise 

As  an  example  for  the  time-varying  signal 
case  wo  consider  a signal  whose  Fourier  trans- 
form ir,  of  a triangular  shape  against  the  ideal 

low  pass  noise  as  in  Ex.  1 . The  asymptotic  J-  i 

Ratios  for  = 0,3  and  0.4  are  plotted  in  Fig.  | 

3,  where  denotes  the  bandwidth  of  the  signal  I 

frequency  spectrum.  As  we  mentioned  in  Part  C, 

See.  II,  the  J-Ratio  remains  at  one  for  1/A>W+W  . 


APPENDIX.  Derivation  of  (24) 

From  (6)  and  (23),  can  be  written  as 

,2A 

= A J N^(f)  df  (Al) 

■ IE 

where  is  a N>'N  Hermitian  symmetric  matrix 
whose  ij-th  element  is  cxp( - j2nAf (i- j ) ) . 

For  N>>1,  we  can  approximate  R.^,\  the  in- 

N 

verse  of 

-1  P 1 

"k  " M 1 dn 

■ 2K  ^ 

It  can  be  shown  that  ^ I>  the  identity 

matrix,  by  usiiig  the  Poisson  sum  formula  in  the 
intermediate  stages. 

Let  S bo  the  signal  component  vector,  i.e. 

= {s  (i  A)  . Then  J^  can  bo  written  as 

1— -N/45 


We  can  then  see  that,  for  N>>1, 


£ |s^(£>] 


where 


N/2-i 

i=-N/2 


Thus  we  get  (24) . 


(A3) 


(A4) 


.4 
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Fig.  2.  Constant  Signal  and  Gauss-Markov  Nois 
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Fig.  3.  Time-Varying  Signal  and  Ideal  Ix>w  Pass  Noise.  Asymptotic  J-Ratio. 
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